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DETAILED ACTION 

1. This Supplemental Office Action supercedes the previous Office Action dated 2 
April 2004 that contained a typographical error in the 35 USC 103(a) prior art 
references, A request for continued examination under 37 CFR 1.114, including the fee 
set forth in 37 CFR 1. 1 7(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1. 1 14, and the fee set 
forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 20 
February 2004 (paper #15) has been entered. Claims 1-26 are currently pending in 
this application. 

Response to Arguments 

2. Applicants arguments filed on 20 February 2004 (paper #15) have been fully 
considered. 

Regarding applicants response to 112(1) rejection : Applicant's argue that the 
prior art submitted in applicants IDS filed on 2 January 2001 contains "voluminous 
teachings" on how components in picture data is recognized and therefor the 
specification need not disclose such techniques in detail. The examiner asserts that 
these various teachings are application specific and do not directly disclose the level of 
skill an artisan would require to realize the claimed invention without undue 
experimentation. The Watanabe 016' patent, for example, is drawn to manipulation of 
three-dimensional information input using a stereoscopic imaging sensor, while the 
Arakawa 450' patent is drawn to the display of a three-dimensional shape model of 
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sensor monitored plant equipment None of the prior art cited by applicants discloses 

specifically how a skilled artisan would perform an image analysis to extract the 

required geometric information from digital picture data, nor specifically how identifying 

and matching the geometric information with component information is performed, (see 

specification: page 8, line 19-23) Applicants arguments are completely silent on - 

specifically how any of the technigues disclosed in the referenced prior art would be 

applied to the method of the claimed invention. Applicants haye also not incorporated 

any of the referenced prior art by reference and have not clarified which elements of the 

claimed invention are novel over the referenced prior art. Applicants arguments appear 

to be an attempt to cure the deficiencies of the specification by incorporating new matter 

(via argument) into the specification after the original filing date. 

Regarding examiner's indication of non-enabled claim elements the MPEP 706 recites: 

"UNIFORM APPLICATION OF THE PATENTABILITY STANDARD 

The standards of patentability applied in the examination of claims must be the same 
throughout the Office. In every art, whether it be considered "complex," "newly 
developed," "crowded," or "competitive," all of the requirements for patentability (e.g., 
novelty, usefulness and unobviousness, as provided in 35 U.S.C. 101, 102, and 103) 
must be met before a claim is allowed. The mere fact that a claim recites in detail all of 
the features of an invention (i.e., is a "picture" claim) is never, in itself, justification for the 
allowance of such a claim. 

An application should not be allowed, unless and until issues pertinent to patentability 
have been raised and resolved in the course of examination and prosecution , since 
otherwise the resultant patent would not justify the statutory presumption of validity (35 
U.S.C. 282), nor would it "strictly adhere" to the requirements laid down by Congress in 
the 1952 Act as interpreted by the Supreme Court. The standard to be applied in all 
cases is the "preponderance of the evidence" test . In other words, an examiner should 
reject a claim if, in view of the prior art and evidence of record, it is more likely than not 
that the claim is unpatentable." 
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Applicants have also referenced pages 15-1 8 of the specification as providing the 
required enablement for the claimed limitations. The examiner has again reviewed 
these passages and respectfully disagrees. For example, specification page 1 8 recites 
that "the evaluation-and-control-unit tries automatically or in interaction with the user to 
identify components in the prepared-sources". This passage implies that the user, in 
the "or" case, and not the evaluation-and-control-unit, actually identifies the components 
in the prepared source. Further, the specification does not disclose how the process 
would "automatically" identify components in the prepared sources. In this case the 
specification is again unclear on how the assignment based on geometric elements is 
achieved, what the geometric elements actually are, and what geometric properties are 
actually assigned the analysis. These passages only further bolster the examiners 
position that the specification for the claimed invention does not provide enablement for 
the claimed limitations. 

The specification further does not disclose, and applicant's response has not 
argued, specifically how the claimed method for deriving a hypotheses for identified 
components in picture data is performed. 

Accordingly, the examiner maintains the 112(1) rejection. 

Regarding applicant's response to 103(a) rejections : Applicant's arguments 
relating to perfection of the priority date are persuasive. (The examiner acknowledges 
the receipt of English translation of priority document, paper # 16) Accordingly, the 
examiner withdraws the previous 103(a) rejection using Marks in view ofAsar, in further 
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view of Crandall as prior art. However, the examiner has now applied new 103(a) 
rejections. (See new 103(a) rejections below) 



Claim Rejections - 35 USC §112 

The following is-a-quotation of the first paragraph of 35 U.S.C. 112: - 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 

3. Claims 1-12 are rejected under 35 U.S.C. 112, first paragraph, as containing 
subject matter which was not described in the specification in such a way as to 
enable one skilled in the art to which it pertains, or with which it is most nearly 
connected, to make and/or use the invention. 

Specifically, in independent claim 1 applicants are claiming an evaluation and 
control unit for comparing component information data with real picture data but the 
specification does not disclose an algorithm or technique for comparing component 
information data with real picture data. Neither the operation of the evaluation and 
control unit nor the process of identifying components in picture data, nor the 
claimed deriving hypotheses, is sufficiently described in the specification to allow one 
skilled in the art to make and/or use the invention . 

While page 4, line 5 of the specification state that the evaluation and control unit 
controls the process of generating the virtual installation model and controls an 
"automatic function" (page 5, line 13), it does not sufficiently describe functional 
operation either the process or the "automatic function". Obviously, one skilled in the art 
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would not be able to create an "automatic function" without specific details on how the 
automatic function operates. Merely stating that the evaluation and control unit 
"processes component data" (page 8, line 8) and "performs image analysis" (page 8, 
line 20) does not provide a sufficient description to allow one skilled in the art to make 
and/or use the invention. Dependent claims inherit these defects. - - 



Claims 13-26 are rejected under 35 U.S.C. 112, first paragraph, because the 
specification, while being enabling for "generating picture data" and "storage to 
store", does not reasonably provide enablement for "comparing picture data to 
identify installation components" (claim 13) or "a processing unit to compare 
components". The specification does not enable any person skilled in the art to 
which it pertains, or with which it is most nearly connected, to make and/or use 
the invention commensurate in scope with these claims. 

Specifically, independent claim 13 claims comparing picture data to identify 
installation components within the picture data but again the specification does not 
disclose an algorithm or technique for performing the comparison or identifying the 
components. 

Independent claim 25 claims a processing unit to compare components but 

does not disclose the process for comparing components sufficient to allow one skilled 
in the art to make and/or use the invention. 

Dependent claims inherit these defects. 
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Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
" forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 

USPQ 459 (1966), that are applied for establishing a background for determining 

obviousness under 35 U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 

4. Claims 1-26 are rejected under 35 U.S.C. 103(a) as being unpatentable over 

U.S. Patent 5,025,347 issued to Marks in view of U.S. Patent 5.640.468 issued to 

Hsu in further view of U.S. Patent 5,552,984 issued to Crandall et al. 

While the specification for the claimed invention is delinquent in the areas cited 

above (see 112(1) rejections), the examiner has made prior art rejections based on the 

limited scope of the information contained in the specification. 

Independent claim 1 is drawn to: 

Generating an image of installation model by: 
Memory (1 st ) for storing picture data 
Memory (2 nd ) for storing component information 
Memory (3 rd ) for storing virtual installation model 

Evaluation and control unit for comparing component information data with real picture data 
Identifying components in picture data as installation components 
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Deriving a hypotheses for identified components in picture data 
Generating respective installation components in virtual installation model 

Regarding independent claim 1 : Marks teaches a system and method of 
generating a 3D (virtual) model from an image (picture) using a computer system. 
The image is of an arrangement of physical objects (components) where primitives 
(geometric) representing a portion of a physical object (a component) in the image are 
specified as a set of parameters that correspond locations in the image model. Marks 
further discloses incorporating a memory space for storing picture data, component 
(object) information. (Abstract Summary of Invention, Figs. 1-7, CL3-L2-10, CL4-L45, 
CL7-L35-67, CL7-L5-50) 

Marks does not explicitly teach comparing component information data with real 
picture data. 

Hu teaches a system and method for identifying components by comparing 
component information data (from a knowledge base) with real picture data and 

incorporating multiple memories for storing picture data and component information. Hu 
further discloses evaluating (deriving a hypotheses via the image system) identified 
components (objects) in the picture data. (Abstract Summary of Invention, Figs. 1-4, 6, 
8a-10b, CL13-L15-CL14-55) 

Hu mentions, but further does not explicitly teach virtual components 
(installation). 

Crandall teaches generating a virtual model of a real system using virtual 
components (installation components) from a library of components representing the 



Application/Control Number: 09/750,673 Page 9 

Art Unit: 2128 

total virtual system (installation). (Abstract Summary of Invention, Figs. 1a&b, 9, CL2- 
L5-11, CL4-L1-29) 

It would have obvious to one of ordinary skill in the art at the time the claimed 
invention was made to modify the teachings of Marks relating to a system and method 
for generating a 3D (virtual) model from an image (picture), with the teachings of Hu - 
relating to identifying components by comparing component information data with 
real picture data from a knowledge base, and to further modify the teachings of Marks 
with the teachings of Crandall relating to generating a virtual model of a real system 
using virtual components (installation components) from a library of components 
representing the total virtual system (installation), to realize a device and method for 
generating a virtual model of an installation. An obvious motivation exists since, as 
referenced by prior art, creating a 3D (virtual) model of physical objects provides more 
efficient detection of discrepancies between the model and the actual image. (See 
Marks Background) 

Regarding dependent claims 2-4 : Hu performs image analysis on picture data as 
previously cited above. Crandall teaches using virtual components (installation 
components) from a library of components representing the total virtual system 
(installation) (CL2-L5-11). Marks teaches objects being located by geometric 
information and multiple window views of picture data and 3D (virtual) model views. 
(CL4-L1 0-25, Figs. 2-7) 

Regarding dependent claims 5-9 : Marks discloses building a 3D (virtual) model 
using the well-known techniques of "dropping", "clicking and dragging", and "rubber 
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banding" in the manipulation of primitives based on geometric information (CL4-L42). 
Marks also teaches the evaluation of structural components (by function) to assign 
primitives (add components) in a 3d (virtual) installation model. (Figs. 9-25, CL3-L29-43, 
CL5-L12-25, CL6-L10-35) 

- Regarding dependent claims -10-12 : Marks teaches-the "automatic" component 

location (CL7-42) and a system incorporating a digital camera, digitized photographs 

(picture data), and a CAD system with memory and a multiple perspective view display. 

(Abstract, Summary of Invention, Figs. 1-7, CL3-L2-10, CL4-L45, CL7-L35-67, CL7-L5- 

50) 

Independent claim 13 is drawn to: 

Generating an image of real installation model by: 
Generating picture data 

comparing component information data with picture data 
Identifying components in picture data as installation components 

Regarding independent claim 13 : As previously cited, Marks teaches a system 
and method o f generating a 3D (virtual) model from an image (picture) using a 
computer system. The image is of an arrangement of physical objects (components) 
where primitives (geometric) representing a portion of a physical object (a component) 
in the image are specified as a set of parameters that correspond locations in the 
image model. (Abstract, Summary of Invention, Figs. 1-7, CL3-L2-10, CL4-L45, CL7- 
L35-67, CL7-L5-50) 
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Marks does not explicitly teach comparing component information data with real 
picture data. 

Hu teaches a system and method for identifying components by comparing 
component information data (from a knowledge base) with real picture data and 

- incorporating multiple-memories for storing picture data and component information. Hu 
further discloses evaluating (deriving a hypotheses via the image system) identified 
components (objects) in the picture data. (Abstract, Summary of Invention, Figs. 1-4, 6, 
8a-10b, CL13-L15-CL14-55) 

Marks mentions, but further does not explicitly teach virtual components 
(installation). 

Crandall teaches generating a virtual model of a real system using virtual 
components (installation components) from a library of components representing the 
total virtual system (installation). (Abstract, Summary of Invention, Figs. 1a&b, 9, CL2- 
L5-11, CL4-L1-29) 

It would have been obvious to one of ordinary skill in the art at the time the 
claimed invention was made to modify the teachings of Marks relating to a system and 
method for generating a 3D (virtual) model from an image (picture), with the 
teachings ofHu relating to identifying components by comparing component 
information data (from a knowledge base) with real picture data, and to further modify 
the teachings of Marks with the teachings of Crandall relating to generating a virtual 
model of a real system using virtual components (installation components) from a 
library of components representing the total virtual system (installation), to realize a 
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device and method for generating a virtual model of an installation. An obvious 
motivation exists since, as referenced by prior art, creating a 3D (virtual) model of 
physical objects provides more efficient detection of discrepancies between the model 
and the actual image. (See Marks Background) 

Regarding dependent claims 14-16 : As also previously cited, Hu teaches image 
analysis on picture data as previously cited above. Crandall teaches using virtual 
components (installation components) from a library of components representing the 
total virtual system (installation) (CL2-L5-11) Marks teaches the manipulation of 
primitives based on geometric information (CL4-L42) and multiple window views of 
picture data and 3D (virtual) model views. (CL4-L10-25, Figs. 2-7). 

Regarding dependent claims 1 7-24 : Marks discloses building a 3D (virtual) model 
using the well-known techniques of "dropping", "clicking and dragging", and "rubber 
banding" in the manipulation (matching) of primitives based on geometric information 
(CL4-L42). Marks also teaches the evaluation of structural components (by function) 
to assign primitives (add components) in a 3d (virtual) installation model. (Figs. 9-25, 
CL3-L29-43, CL5-L12-25, CL6-L10-35) It further would have been obvious, and 
necessary, to "select" and "drag" installation components since all modern GUI based 
CAD system operate in this manner. Marks further teaches "automatic" component 
location (CL7-42) and a system incorporating a digital camera, digitized photographs 
(picture data), and a CAD system with memory and a multiple perspective view display. 
(Abstract, Summary of Invention, Figs. 1-7, CL3-L2-10, CL4-L45, CL7-L35-67, CL7-L5- 
50) 
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Regarding claims 25 and 26 : Claims 25 and 26 merely claim the virtual model of 
a facility that includes the same limitations as disclosed in claims 1-24. Claims 25 and 
26 are therefore rejected using the same reasoning as previously cited above. 

Conclusion 

5. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure, careful consideration should be given prior to applicants 
response to this Office Action. 

U.S. Patent 5,988,862 issued Kacyra et al teaches component modeling from picture 
data. 

L/.S. Patent 5,894,310 issued to Arsenault et al teaches virtual modeling of systems. 
U.S. Patent 5,812,394 issued to Lewis et al teaches virtual components and installation. 
U.S. Patent 4,937, 768 issued to Carver et al teaches virtual system modeling. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Fred Ferris whose telephone number is 703-305-9670 
and whose normal working hours are 8:30am to 5:00pm Monday to Friday. 
Any inquiry of a general nature relating to the status of this application should be 
directed to the group receptionist whose telephone number is 703-305-3900. 
The Official Fax Numbers are: 

Official ' (703) 872-9306 

7ew&, Patent Examiner 
Simulation and Emulation, Art Unit 2128 
U.S. Patent and Trademark Office 
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